MEMORANDUM September 13, 2017 


TO: Lance Menster 
Director, Curriculum and Development Department 


FROM: Carla Stevens 
Assistant Superintendent, Research and Accountability 


SUBJECT: THE IMPACT OF THE HOUSTON INDEPENDENT SCHOOL DISTRICT READ 
HOUSTON READ MENTORING PROGRAM ON FIRST-GRADE STUDENTS’ 
READING LEVELS, 2016-2017 


Read Houston Read is a first-grade mentoring program being implemented in the Houston 
Independent School District (HISD) with support from the Barbara Bush Houston Literacy 
Foundation (BBHLF) since the 2014—2015 school year. As a supplement to the district’s 
Literacy By 3 initiative, RHR establishes a foundation for the development of students’ reading 
skills. Community and business volunteers use read-aloud strategies in face-to-face or online 
environments to boost students’ reading enjoyment. The purpose of this evaluation was to 
determine the RHR impact on students’ reading levels. 


Key findings include: 

e A larger proportion of Spanish learners (71%) compared to English learners (44%) were 
assessed during beginning (BOY), middle (MOY), and end of year (EOY) assessment 
windows. 

e There was an increase in the percentage of English and Spanish learners who met 
expected or advanced development reading levels using both face-to-face and online 
modalities when each assessment window, (BOY, MOY, EOY) was considered 
independently. 

e A larger percentage of online English learners met expected and advanced development 
reading levels compared to their face to-face peers and a larger percentage of online 
Spanish learners met expected and advanced development reading compared to their face- 
to-face peers except non-economically-disadvantaged and limited English proficiency 
learners. 

e By year’s end, using a repeated measures design, more face-to-face English and more 
online Spanish learners showed reading growth compared to their respective peers. 


Further distribution of this report is at your discretion. Should you have any further questions, 
please contact me at 713-556-6700. 
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BUREAU OF PROGRAM EVALUATION 


The Impact of the Houston Independent School District Read Houston Read 
Mentoring Program on First-Grade Students’ Reading Levels, 2016-2017 


By Ted D. Serrant, Ph.D. 


The purpose of this evaluation was to assess the impact of the Houston Independent School District’s (HISD) Read 
Houston Read (RHR) on the reading levels of first-grade students. Students were assessed using Benchmark 
Reading Record (BRR) that was aligned to leveled books and Fountas and Pinnell Reading Levels. Descriptive 
Statistics were used to develop profiles of both RHR English and Spanish students as well as the proportion of 
students who met expected or advanced reading levels at the beginning (BOY), middle (MOY), and the end of the 
year (EOY), 2016-2017, who used face-to-face or online modalities. These students were disaggregated by key 
educational and demographic variables. Results indicate that a higher proportion of online English students met 
expected or advanced reading compared to their face-to-face peers and a higher percentage of online Spanish 
students met expected or advanced reading levels compared to their peers in the face-to-face mode, except non- 
economically-disadvantaged and limited English proficiency (LEP) students. A repeated-measures design was 
used to determine students’ reading growth between the BOY and EOY. By year’s end, based on the repeated 
measures results, more face-to-face English and online Spanish students showed reading growth compared to their 
respective peers. Systematic and consistent assessment of all RHR students within the designated windows is 
recommended to better determine program impact. 


Business and community volunteers use read-aloud 
strategies in face-to-face or online environments to 
boost children’s reading enjoyment. These volunteers 
provide thirty minutes to one hour of read aloud to 
students using books that these students or mentees 
enjoy reading and that are at the mentees’ grade levels. 

HISD provided training for all volunteer mentors. 


Background 


Read Houston Read (RHR) is a first-grade mentoring 
program initiated during the 2014—2015 school year in 
the Houston Independent School District (HISD) with 
support from the Barbara Bush Houston Literacy 
Foundation’s (BBHLF) Blueprint for Community 


Action. It is administered as a supplement to the 
district’s Literacy By 3 initiative and is aligned to the 
goals and recommendations of BBHLF. The program 
establishes a foundation for the development of 
students’ reading skills. Among its goals, the 
Foundation seeks to ensure that every child who enters 
kindergarten is ready to read and that every child reads 
proficiently by the end of third grade. BBHLF strategies 
include a recommendation to “promote existing and 
new initiatives to read books to children across all 
formats (traditional and digital)’ (Barbara Bush 
Houston Literacy Foundation, 2014, p. 31). RHR is 
built on the assumption that the acquisition of basic 
reading skills requires the involvement and assistance 
of a wide range of volunteers and that reading aloud to 
children improves their reading levels and proficiency. 


Volunteers learned “how they could share the magic of 
a book by reading to a child, engage in fun activities 
that directly relate to the reading, and listen to a child 
read as they share a book” (HISD, 2015a). Mentors 
worked with the same students for an entire school year. 
This strategy builds important relationships with lasting 
impacts on students’ self-esteem, their ability to learn 
and, ultimately on their academic success (HISD, 
2015a). 

Mentors were assigned to first-grade students during 
the school year to reinforce the reading progress their 
teachers initiated in the classroom through uniform, 
proven methods (HISD, 2015a). Seventy-three schools 
volunteered to implement the RHR mentoring program 
for the 2016—2017 school year. Students from forty-one 
of the schools had face-to-face mentoring, while 
students in 23 schools had online mentoring and 
students in nine schools had both face-to-face and 


online mentoring. The list of schools participating in the 
program is in Appendix A (p. 9). 

In the online mode, students are read to using 
computer and audio devices unlike the face-to-face 
mode in which volunteers read to students in-person. 
The purpose of this evaluation was to determine the 
comparative reading levels and reading growth of 
students in the face-to-face and online version of RHR. 
The evaluation was designed to answer the following 
questions: 

1. What is the demographic and educational profile 
of RHR student participants for the 2016-2017 
school year? 

2. How did the reading levels of RHR participants, 
overall, change during the 2016-2017 school 
year? 

3. How did the reading levels of RHR participants 
compare between the face-to-face and online 
modalities? 

4. To what extent did RHR students experience 
reading growth during the 2016-2017 school 
year? 


Literature Review 


According to the 1985 report by the United States 
Commission on Reading, Becoming a Nation of 
Readers, “the single most important activity for 
building knowledge for their eventual success in 
reading is reading aloud to children” (Anderson, 
Heibert, Scott & Wilkinson, 1985). Research on 
reading aloud or reading to children has been shown to 
have social and emotional benefits, positive impacts on 
their language and literacy development, and 
motivation to read (Swanson, Vaughn, Petscher, 
Heckert, Cavanaugh, Kraft & Tackett, 2011). 

A meta-analysis of 18 studies confirm significant, 
positive effect of read-aloud instruction on the language 
phonological awareness, print concept, comprehension, 
and vocabulary of children. Notwithstanding, the read- 
aloud intervention type accounted for only a small 
amount of outcome variance (Swanson et al., 2011). 
The read-aloud instruction included dialogic reading, 
repeated reading of stories, story reading with limited 
questioning before, during, and/or after reading, 
computer-assisted story reading, and story reading with 
extended vocabulary activities (Swanson et al., 2011). 
Scarborough and Dobrich found that reading aloud 
accounts for only 8% of the variance in reading ability 
in primary grade (cited in Lane & Wright, 2007). 

Lippman (1997) studied 45 New Jersey first-grade 
students to determine the impact of early read-aloud on 
their reading success in the first grade. Students’ 
aptitude test scores based on teacher observations and 
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test scores from the MacMillan/McGraw-Hill “A New 
View” readings series were analyzed. Questionnaires 
were administered to parents with an 84% response rate 
and students were grouped in rich and poor learning 
experiences. T-test results showed a. statistically 
significant 24-point difference in the mean reading 
performance of samples, in favor of the literacy-rich 
group. There was, however, no strong evidence that 
reading to children at an early age improved reading 
success in the first grade. 


Method 


This is a comparative evaluation of the face-to-face 
and online modes of RHR delivery based on students’ 
Benchmark Running Record (BRR) for the 2016-2017 
school year. The evaluation used descriptive and 
inferential statistics to measure and compare the 
students’ reading levels in face-to-face and online 
modalities, as well as changes in their reading levels at 
the beginning (BOY), middle (MOY), and end of the 
school year (EOY). 

Descriptive and inferential statistical analyses 
included a repeated measures design to compare the 
beginning (BOY) and end-of-year (EOY) reading 
levels of first-graders who were involved in the RHR 
program, and the use of the Wilcoxon signed-rank test 
to determine any significant differences in changes in 
these levels for both English and Spanish students. The 
study also estimated the percentage changes in the 
proportion of RHR students’ reading levels at the BOY, 
MOY, and EOY disaggregated by key demographic 
and educational variables. 


Data Collection 


The list of schools participating in the RHR program 
was retrieved from the Elementary Curriculum and 
Development Department webpage on the HISD 
website. The list contained participating schools by 
modality. The reading levels based on teacher-reported 
Benchmark Running Record was collected from the 
Student Assessment Department’s SharePoint site as 
text (.txt) files. These were later exported into an 
Microsoft Excel spreadsheet for each first-grade 
Spanish and English learner, by modality, who were 
assessed at the beginning (BOY) (August 22-— 
September 23), middle (MOY) (December 5—January 
13), and end (EOY) (May 1—May 26) of the 2016-2017 
school year. Benchmark Running Records (BRR) is a 
formative reading assessment instrument based on 
Fountas and Pinnell Guided Reading levels (Scholastic 
Inc., 2010). These alphabetic measures, which are 
linked to leveled reading books have been categorized 
for interpretation purposes into More Development 


Needed (MDN), Meets Expectations (ME), and 
Advanced Development (AD) by the Curriculum 
Department. Details are in Appendix B, Table 3, p. 10. 
Teachers were trained to assess students’ reading levels 
using Benchmark Running Record prior to the 
commencement of the 2016-2017 school year. Key 
educational and demographic data from the Research 
and Accountability Department Public Education 
Information System (PEIMS) Microsoft Access 
database were also collected and linked to each 
students’ reading level for aggregated and 
disaggregated analyses. 


Data Analysis 


Students’ demographic and academic data as well as 
teacher and school attribute data were uploaded into 
International Business Machine (IBM) Statistical 
Packages for Social Scientists (SPSS) for statistical 
analyses. SPSS is a statistical software for the analyses 
of descriptive and inferential data. The analyses 
included the profile of students in the program, their 
reading levels, and growth. A simple difference in 
differences approach was used to determine changes in 
students’ English and Spanish reading levels as a group 
and disaggregated by key educational and demographic 
variables. A repeated measures design was also used to 
determine the growth in the reading levels of students 
who were assessed at the BOY and the EOY. Repeated 
measures design uses the same student sample across 
time as a robust analysis to determine reading growth. 
Finally, a Wilcoxon signed-ranked test! was used to 
determine the significant differences in the reading 
level between the BOY and EOY. Data results are 
presented in charts and tables. 


Limitations 


Given the complex nature of reading and the multiple 
reading programs being implemented in HISD, it is 
unlikely that any growth or changes can be attributed 
solely to RHR. This report did not control for those 
other programs to which students in this sample may 
have been exposed. The assumption is that they all have 
been exposed to the same programs. The non- 
quantifiable aspects of the RHR may not have been 
captured in this report, however, a substantial part was 
captured in the 2014-2015 evaluation report (see 
(HISD, 2015b) 

A critical aspect of the relationship between read- 
aloud and literacy is the instruction or intervention 


! The Wilcoxon signed-rank test is the nonparametric test equivalent to the dependent t-test. 
As the Wilcoxon signed-rank test does not assume normality in the data, it can be used when 
this assumption has been violated and the use of the dependent t-test is inappropriate. It is 
used to compare two sets of scores that come from the same participants. This can occur 
when we wish to investigate any change in scores from one time-point to another, or when 
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strategy used — dialogic and text talk, which this 
evaluation does not capture. Actual classroom 
observations would have provided useful data from 
which to determine how these are transacted in the 
classroom and the relationship between these strategies 
and student performance. 

Participation in RHR is voluntary and sustained 
commitment to the administration of the BRR during 
the assigned windows may be a challenge, which may 
have negative consequences for program fidelity and 
measuring the full impact of the program on student 
reading levels. 


Result 


What is the demographic and educational profile of 


RHR student participants for the 2016-2017 school 
year? 


An average of 2,661 HISD English first-grade 
student participants had a BRR reading assessment for 
the beginning (BOY) (2,568), middle (MOY) (2,084), 
and end (EOY) (3,330) of the 2016-2017 school year. 
As at the EOY, 1,973 (59.7%) and 1,327 (40.1%) 
students participated in the face-to face or online 
modalities, respectively. The educational and 
demographic distribution of students in the study 
sample were comparable as shown in Table 1 and 
Table 2. 


Table 1. Educational and Demographic Profile of English Learners 
RHR Participants by Mode, HISD 


Education/ n Face-to-Face Online 
Demographic (n=1,973) (n=1,327) 
Female 1,597 48.0 49.0 
Gender Male 1,703 52.0 51.0 
Econ. No 531 16.0 16.2 
Disadv. Yes 2,769 84.0 83.3 
: No 918 273 28.6 
ae Rich: Yes 2,382 72.7 714 
Special No 3,180 95.7 97.3 
Education Yes 120 4.3 2:7 
No 2,202 67.0 66.3 
eed Yes 1,098 33.0 33.7 
i English 2,084 63.2 63.1 
Spoame : Spanish 1,040 31.2 32.0 
gua’ Other 176 5.6 49 
No 3,081 93.2 93.6 
ile Yes 219 6.8 6.4 
Asian 126 BL. 4.0 
a Black 1150 34.0 36.2 
Beery Hispanic 1808 55.9 53.1 
White 1723 5.2 53 


Source: Research and Accountability PEIMS Microsoft Access database, Fall Snapshot, 
2016-2017 


individuals are subjected to more than one condition. (https://statistics.laerd.com/spss- 
tutorials/wilcoxon-signed-rank-test-using-spss-statistics.php) 


An average of 2,560 Spanish RHR _ first-grade 
participants had Benchmark Running Record reading 
assessment data for the BOY (2,124), MOY (2,216), 
and EOY (2,341). As at the EOY, 1,369 (58.5%) and 
972 (41.5%) students participated in the face-to-face 
and online RHR, respectively. As shown in Table 2, the 
demographic and educational profile of RHR Spanish 
learners by mode were comparable. 


Table 2. Educational and Demographic Profile of Spanish Learners 
RHR Participants by Mode, HISD 


Education/Demographic n Face-to-Face Online 
(n= 1,369) (n= 972) 
Gender Female 1,157 50.2 48.4 
Male 1,184 49.8 51.6 
Econ. No 185 8.0 Td 
Disadv. Yes 2,156 92.0 92.3 
At Risk No 167 8.3 a) 
Yes 2,174 91.7 94.5 
Special No 2,273 97.2 97.0 
Education Yes 68 2.8 3.0 
LEP No 404 19.1 14.6 
Yes 1,937 80.9 85.4 
Home English 361 16.9 13.4 
Language Spanish 1,970 82.8 86.1 
Other 10 50.0 50.0 
G/T No 2,190 94.5 92.2 
Yes 151 5:5 7.8 
Ethnicity Asian 8.0 0.3 0.4 
Black 146 5.3 7.6 
Hispanic 2,164 93.4 91.0 
White 16 0.7 0.6 


Source: Research and Accountability PEIMS Microsoft Access Database of Fall Snapshot, 
2016-2017 


How did the reading levels of RHR participants, 
overall, change during the 2016-2017 school year? 


This report focused on the overall reading levels of 
first-grade students involved in the mentoring program 
regardless of RHR modality. It looked at the growth and 
finally it looked at the performance levels by key 
demographic characteristics. 

Figure 1 and Figure 2 show the reading levels of 
students irrespective of the RHR mentoring mode (face- 
to-face or online) for English and Spanish learners at 
BOY, MOY, and EOY. For the purposes of this report, 
English learners are instructed in English and Spanish 
learners are instructed in Spanish. Figure 1 shows a 
decrease in the percentage of RHR English learners 
needing more reading development (MDN) from 63.0 
percent to 53.5 from the BOY to the EOY. The 
proportion of English learners who met expected or 
advanced development reading levels increased from 
36.9 percent at the BOY to 46.5 percent at the EOY, a 
difference of 9.6 percentage points. At the MOY, 
however, 52.1 percent of the students met expected or 
advanced development reading levels, compared to 
47.9 percent of those who still required more 
development in reading. 
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Figure 1. Reading levels of RHR English Learners, HISD, 2016-2017 


MDN = More Development Needed; ME = Met Expectations; Adv=Advanced 
Development 


Figure 2 shows the reading level of first-grade 
Spanish-learners at the beginning (BOY), middle 
(MOY), and end of year (EOY). The percentage of 
students who needed more reading development 
(MDN) decreased from 63.9 percent to 44.3 percent 
between the BOY and EOY, 2016-2017. The 
percentage of students who met expected (ME) or 
advanced (AD) reading levels increased from 36.1 
percent (BOY) to 55.7 percent at the EOY, an increase 
of 19.6 percentage points. Overall, there was a larger 
increase in the proportion of RHR Spanish learners 
(19.6 percentage points) compared to English learners 
(9.6 percentage points) who met expected (ME) or 
advanced (AD) reading levels between the BOY and 
EOY. 
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Figure 2. Reading levels for RHR Spanish learners, HISD, 2016-2017 


MDN = More Development Needed; ME = Met Expectations; Adv=Advanced 
Development 


How did the reading levels of RHR participants 


compare between the face-to-face and online 
modalities? 


RHR student reading levels were compared by 
modalities to determine difference in performance at 
the BOY, MOY, and EOY. Figure 3 and Figure 4 
present the findings. 
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Figure 3. Comparative Reading Levels of RHR English Learners By 


modalities, HISD, 2016-2017 


MDN = More Development Needed; ME = Met Expectations; AD =Advanced 
Development 


While the proportion of RHR face-to-face English 
learners who needed more development in reading 
decreased from 73.1 percent to 60.4 percent (a 
difference of 12.7 percentage points) between the BOY 
and EOY, the proportion of RHR online students who 
needed more development in reading decreased from 
70.5 percent to 50.6 percent, a difference of 19.9 
percentage points for the same period as shown in 
Figure 3. 

The proportion of RHR _ face-to-face English 
learners who met reading expectations or read at an 
advanced level at the BOY and EOY increased from 
26.9 percent to 39.6 percent, a difference of 12.7 
percentage points, while the proportion of their RHR 
online peers reading at the expected or advanced level 
increased from 29.5 percent to 49.4 percent, a 
difference of 19.9 percentage points for the 
corresponding period. Based on Figure 3, a larger 
proportion of English learners using the RHR online 
mode appeared to have been reading at the expected 
(ME) and advanced (AD) level at the EOY when 
compared to their face-to-face peers given a similar 
starting point at the BOY. 
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Figure 4. Comparative Reading Levels of RHR Spanish Learners by 


Modality at the BOY, MOY and EOY, HISD, 2016-2017 


MDN = More Development Needed; ME = Met Expectations; AD. =Advanced Reading 


As shown in Figure 4, the proportion of face-to-face 
RHR Spanish learners who needed more reading 
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development decreased from 65.9 percent (BOY) to 
52.3 percent (EOY), a difference of 13.6 percentage 
points. The proportion of RHR online students who 
needed reading development also declined from 61.2 
(BOY) to 51.6 percent (EOY), a difference of 9.6 
percentage points. Conversely, the proportion of RHR 
face-to-face Spanish learners who met expected or 
advanced reading levels at the BOY and EOY were 34.1 
percent and 47.7 percent, respectively, a difference of 
13.6 percentage points. The proportion of RHR online 
students who met expected or advanced reading levels 
increased from 38.7 percent (BOY) to 48.4 percent 
(EOY), a difference of 9.7 percentage points. A larger 
proportion of RHR _ face-to-face Spanish learners 
appeared to have improved their reading over their 
online counterparts between the BOY and EOY as 
shown in Figure 4. 

The BRR data was further disaggregated to determine 
the reading levels of RHR English face-to-face and 
online learners by selected student educational and 
demographic variables: gender, economically 
disadvantaged, at risk, limited English proficiency 
(LEP), ethnicity, and home language. Table 4 through 
Table 10, Appendix C, pp. 11—13 provides details of 
the findings. 

Similar data for RHR Spanish learners are available 
in Table 11 through Table 17, Appendix D, pp 14-16. 
The data show that the percentage of RHR English 
learners reading at the advanced development level 
increased between the BOY and EOY by students’ 
gender (Table 4), economic status (Table 5), at-risk 
status (Table 6), LEP (Table 7), and ethnicity (Table 
8). 

The largest proportion of English learners by 
ethnicity who used face-to-face modes and who met 
expected or advanced development reading levels at 
year’s end were Black (42.7%), compared to White 
students (53.0%) among online students (Table 8, p.12). 
The highest percentage of RHR English learners by 
home language (40.9%) who met expected or advanced 
reading levels among face-to-face students were 
English learners with English home language compared 
to English learners with Spanish home language for 
online mentees (52.5%) (Table 9, p. 12). 

A largest proportion of RHR face-to-face Spanish 
Learners met combined expected or advanced reading 
levels by gender (Table 11), economic status (Table 
12), at risk, (Table 13), and LEP (Table 14). When 
disaggregated by ethnicity, Hispanic Spanish face-to- 
face learners were the largest proportion of students 
who met expected or advanced reading levels (47.6%) 
compared to White for online Spanish learners (83.4%) 
(Table 15). The largest proportion of Spanish learners 
by home language who used the face-to-face mode and 
who met expected or advanced development reading 
levels were those whose home language was English 


(48.9%) compared to Spanish learners with Other home 
languages who used online modes (80.0%) (Table 16). 

The data revealed increased percentages in the 
proportion of both English and Spanish learners who 
met expected (ME) or advanced (AD) reading levels 
over the year, and a reduction in the proportion of 
learners who required more development (AD) when 
the panels (BOY, MOY, and EOY) were considered 
independently. 


To what extent did RHR students experience 
reading growth during the 2016-2017 school year? 


Results of the paired sample analysis for RHR 
English learners are shown in Figure 5. Details are on 
Table 10, Appendix C, p. 13 
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Figure 5. Proportional Difference in Reading Levels of RHR English 


Learners by Mode from BOY to EOY, HISD, 2016-2017 


MDN = More Development Needed; ME = Met Expectations; AD. =Advanced Reading 


Both face-to-face and online groups of RHR students 
showed reading growth (ME & AD) by the end of the 
school year. A larger change in the proportion of 
English RHR face-to-face learners compared to their 
online peers, (16.4 v. 9.3 percentage points, 
respectively) met expected (ME) or advanced (AD) 
reading levels at the EOY. 

Figure 6 shows the paired sample analysis depicting 
reading growth for RHR Spanish learners. Details are 
in Table 17 in Appendix D, p.16. 


100 
80 
60 


-20 10 -7.6 


Percentage Point Difference 


™ Face-to-Face Difference ™ Online Difference 


Figure 6. Proportional Difference in Reading Levels of RHR Spanish- 


learners by Mode from BOY to EOY, HISD, 2016-2017 
MDN = More Development Needed; ME = Met Expectations; AD. =Advanced Reading 
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As a group, the change in the proportion of RHR 
Spanish learners who needed more _ reading 
development decreased for both to face-to-face and 
online modes during the school years (Figure 6). A 
higher change in the proportion of RHR Spanish- 
speaking face-to-face students compared to their online 
peers met expected or advanced reading levels (10.1 v. 
7.7 percentage points). 

Reading levels were ranked to determine the effect of 
RHR using MDN as 1, ME as 2 and AD as 3 and a 
cohort of students who were had both BOY and EOY 
BRR_ assessments. Wilcoxon  signed-ranked _ test 
showed statistically significant changes in the reading 
levels of a cohort of RHR English learners from BOY 
to EOY who used face-to-face (Z = -8.00, p = .000) and 
online (Z= -3.37, p = .001) modalities. Statistically 
significant changes were also found for the cohort of 
RHR English learners who used face-to-face modality. 
The mean reading level rank was 1.68 (BOY) and 1.35 
(EOY) for face-to-face modality and 1.61 (BOY) and 
1.43 (EOY) for the online mode. 

Wilcoxon signed-ranked test showed statistically 
significant changes in the reading levels of a cohort of 
RHR Spanish learners from BOY to EOY who used 
face-to-face (Z = -2.38, p = .017) and online (Z= -3.72, 
p = .001) modalities. Statistically significant changes 
were also found for the cohort of RHR English learners 
who used face-to-face. The mean reading level rank 
was 1.56 (BOY), and 1.43 (EOY) for face-to-face 
modality and 1.61 (BOY) and 1.39 (EOY) for the online 
mode. 

Overall, RHR English-learners appeared to have 
outperformed the Spanish learners by year’s end based 
on the proportion who met expected and advanced 
reading levels, except for their peers at the online 
advanced reading level. However, regardless of 
modality or language, the reading levels of a cohort of 
RHR students for both BOY and EOY data seemed to 
decline between BOY and EOY. This is based on the 
results of the repeated-measures design and the 
Wilcoxon signed-rank test that showed the declines 
were Statistically significant (p.<.001). 


Discussion 


The purpose of the evaluation was to assess the 
impact of the Read Houston Read (RHR) on the reading 
levels of first-grade student participants. The data 
showed an overall increase in the percentage of both 
English and Spanish learners who met expected (ME) 
or advanced (AD) reading levels using both face-to- 
face and online modalities when each assessment 
window (BOY, MOY, EOY) was considered 
independently. The data showed a reduction in the 
percentage of students who needed more development 


in reading by the end of the school year, 2016-2017. 
However, the percentages of RHR students who were 
assessed using BRR varied among the BOY, MOY and 
EOY. The number of RHR students who were assessed 
at all three periods were 44 percent and 71 percent of 
all English and Spanish learners, respectively. There 
appeared to be greater commitment to assessing 
Spanish learners within the designated assessment 
window for the 2016-2017 HISD school year. The 
inconsistency in adhering to the assessment window 
guidelines may be making it difficult to fully assess 
growth in RHR students’ reading levels. 

When compared by mode, a larger proportion of 
online English learners met expected or advanced 
reading levels compared to their face-to-face 
counterparts. Likewise, a larger proportion of online 
Spanish learners met expected or advanced reading 
levels compared to their face-to-face peers. When 
disaggregated by key demographic and educational 
factors, that is, gender, economic status, non-at-risk, 
and non-LEP, a higher proportion of online RHR 
English learners read at the advanced reading level 
compared to their face-to-face peers. Conversely, a 
higher RHR Spanish learners using face-to-face modes 
met expected or advanced reading levels by female, 
non-economically-disadvantaged, and LEP student 
groups. 

By ethnicity, the largest proportion of English 
learners who met expected or advanced reading levels 
at the EOY were Black face-to-face mentees, and White 
online students. For Spanish learners, it was Hispanic 
face-to-face mentees and White online mentees. When 
home language was considered, the largest proportion 
of face-to-face English learners who met expected or 
advanced reading levels at EOY were those whose 
home language was English followed closely by 
Spanish, and for online students it was learners whose 
home language was Spanish. For face-to-face Spanish 
learners, it was mostly students with English home 
language who met expected and advanced reading 
levels. For online Spanish learners, it was mostly 
students with languages other than English or Spanish 
(Other), who met expected or advanced reading level at 
the EOY. 

Overall, a larger proportion of online English and 
Spanish learners met expected or advanced reading 
levels compared to their face-to-face peers. A higher 
proportion of Spanish and English RHR students who 
used face-to-face modalities met expected or advanced 
reading levels compared to their online peers when a 
repeated measure design was used to determine reading 
growth. However, when the cohort of all students 
assessed at the BOY and EOY, was compared, there 
was a decline in the mean reading levels of English and 
Spanish Learners in both the face-to-face and online 
modes based on the Wilcoxon-signed rank test. 
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Greater effort may be required to get a larger number 
of students assessed, systematically and within the 
assigned assessment window. Failure to do so may be 
undermining the district’s ability to determine the true 
impact of the RHR program. 
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Appendix A 


List of HISD Read Houston Read (RHR) Schools, 2016-2017 


Face-to-Face (F2F) 


1. Alcott ES 


2. Ashford ES 


3. Benavidez ES 


4. Blackshear ES 


5. Bonham ES 


6. Bruce ES 
7. Burnet ES 
8. Cook ES 


9. Cunningham ES 


10. Davila ES 


11. Elmore ES 


12. Elrod ES 


13. Foerster ES 


14. Foster ES 


15. Fondren ES 


16. Garcia ES 


26. Kennedy ES 10. Garden Oaks Montessori 
PK-8 

27. Law ES 

11. Harris, JR ES 
28. Lewis ES 

12. Lyons ES 
29. McNamara ES 

13. Lovett ES 
30. Memorial ES 

14. Mading ES 
31. Milne ES 

15. Montgomery ES 
32. Northline ES 

16. Piney Point ES 
33. Peck ES 

17. Reynolds ES 
34. Pleasantville ES 

18. Scarborough ES 
35. Pugh ES 

19. Seguin ES 
36. Reagan K-8 

20. Sherman ES 
37. Ross ES 

21. Wesley ES 
38. Sinclair ES 

22. White ES 
39. Stevens ES 

23. Whittier ES 
40. Thompson ES 

Both (F2F & Online) 

41. Tierina ES 1. Bastian ES 


Online 


17. Gregory-Lincoln ES 


Arabic Immersion 


2. Lockhart ES 


18. Grissom ES 


3. Martinez, R. ED 


19. Gross ES 


4. McGowen ES 


20. Hartsfield ES 


5. Petersen ES 


21. Helms ES 


6. Piney Point ES 


22. Highland Heights ES 


7. Southmayd ES 


23. Hilliard ES 


8. Walnut Bend ES 


24. Hobby ES 


9. Woodson PK-8 


25. Jefferson ES 


2. Benbrook ES 
3. Berry ES 

4. Braeburn ES 
5. Brookline ES 
6. Burrus ES 

7. Bush ES 

8. Coop ES 

9. Crockett ES 


Source: Read Houston Read Webpage; HISD Website, http://www.houstonisd.org/readhoustonread 
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APPENDIX B 
Table 3. Houston ISD Running Record Benchmarks 
BOY MOY EOY 
GRADE 
FP FP FP FP FP FP FP FP FP 
LEVEL | LEVEL | LEVEL | LEVEL | LEVEL | LEVEL | LEVEL | LEVEL | LEVEL 
No benchmark; 
ends eee a aS Be ey, Ez 
BE —_aicenosticfoesline ony a ie co ie 
D-E F-Z E-G H-Z I-J K-Z 
D-E F-Z E-G H-Z 1-J K-Z 
J-K L-Z L M-Z M-N O-Z 
J-K L-Z L M-Z M-N O-Z 
N O-Z O P-Z P-Q R-Z 
N O-Z O P-Z P-Q R-Z 
Q R-Z S T-Z S-T U-Z 
Q R-Z S T-Z S-T U-Z 
U V-Z Vv W-Z V-W X-Z 
U V-Z V W-Z V-W X-Z 


Meeting Expectations 


Advanced Development 


Source: HISD Pre-Approved Performance Tasks, 2016-2017, Houston ISD Curriculum, p. 2 
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Appendix C: RHR English Learners 


Table 4. Comparative Reading Levels of RHR English Learners by Mode and Gender, HISD, 2016-2017 


Face-to-Face : Online 
BOY MOY EOY ee BOY MOY EOY eo 
Female Male Female Male Female Male Female Male Female Male Female Male Female Male Female Male 
(n=693) (n=893) (n=763) (n=881) (n=947) (n=1,026) (n=518) (n=527) (n=167) (n=203) (n=650) (n=677) 
MDN 723.7 72.5 64.2 60.2 59.0 60.1 60.0 69.1 37:5, 47.3 50.3 50.8 
ME 15.4 13.9 20.4 22.4 18.9 17.7 14.6 17.8 375 28.6 20.5 20.7 
AD 10.8 13.6 15.3 17:5 22.1 22.1 12.3 13.1 25.0 24.1 29.2 28.5 


Table 5. Comparative Reading Levels of RHR English Learners by Mode and Economic Status, HISD, 2016-2017 


Face-to-Face Online 
BOY MOY EOY Difference BOY/EOY BOY MOY EOY Difference BOY/EOY 
pee Non- Econ. Non- Econ. Non- Econ. Non- Econ. Non- Econ. Non- Econ. Non- Econ. Non- Econ. 
bb Econ. Disadv. Econ. Disadv. Econ. Disadv. Econ. Disadv. Econ. Disadv. Econ. Disadv. Econ. Disadv. Econ. Disadv. 
Disadv. Disadv. Disadv. Disadv. Disadv. Disady. Disady. Disady. 
(n=1338) (n=1,512) (n=232) (n=1,412) (n=316) (n=1,657) (n = 884) (n= 1045) (n= 57) (n = 313) (n=215) (n=1,112) 
MDN 66.7 73.9 65.9 61.4 60.8 59.4 73.9 69.9 42.1 50.2 49.3 50.8 
ME 17.2 14.3 13.8 22.7 16.8 18.6 18.0 17.4 21.1 27.2 19.5 20.8 
AD 16.1 11.8 20.3 15.9 22.5 22.0 oa 8.1 12.7 36.8 22:7 31.2 28.4 en 


Face-to-Face Online 
BOY MOY EOY Difference BOY MOY EOY Difference 

tea BOY/EOY BOY/EOY 
a Non-At AtRisk | Non-At  AtRisk | Non-At At Non-At At Non-At At Non-At At Non-At At Non-At At 

Risk Risk Risk Risk Risk Risk Risk Risk Risk Risk Risk Risk Risk Risk 

(n=414) —— (n=1098) (n=485) (n=1159) (n=538) (n=1435) (n=308) (n=737) (n= 101) (n=269) (n=380) (n=947) 
MDN 70.8 74.0 64.1 61.2 62.3 58.6 711 70.3 44.6 50.6 51.1 50.4 
ME 14.5 14.7 18.4 22.8 177 18.5 17.9 174 25.7 26.4 20.5 20.6 
AD 147 114 175 16.0 20.1 22.9 11.0 123 29.7 23.0 28.4 29.0 
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Table 7. Comparative Reading Levels of RHR English Learners by Mode and LEP Status 


English LEP Face-to-Face Online 

Reading BOY MOY EOY Difference BOY/EOY BOY MOY EOY Difference BOY/EOY 

as Non-LEP LEP Non-LEP LEP Non-LEP LEP Non-LEP LEP = Non-LEP LEP Non-LEP LEP Non-LEP LEP Non-LEP LEP 
(n= 1,103) (n = 409) (n=1196) (n=448) (n=1322) (n=651) (n= 728) (n =317) (n=237) (n=133) (n=880) (n=447) 

MDN 72.1 75.8 62.7 60.3 39.5 59.9 68.8 74.4 37.5 S11 50.8 50.1 

ME 14.4 15.2 20.6 23.9 19.4 16.0 18.8 14.5 37.5 25.6 20.2 213 

AD 13.5 9.0 16.7 15.8 21.1 24.1 12.4 11.0 25.0 23.3 29.0 28.6 


Table 8. Comparative Reading Levels for RHR English Learners by Mode and Ethnicity, HISD, 2016-2017 


Face-to-Face Online 
Reading BOY MOY EOY BOY MOY EOY 
Level Black Hispanic White Black Hispanic White Black Hispanic White Black Hispanic White Black Hispanic White Black Hispanic White 
(n= 634) (n= 764) (n=66) = (n=711) (n=798) (n=83) — (n=670) (n=1103) (n=103) (n= 360) (n= 589) (n=57) — (n=137) (n=190) (n=21) —— (n=480) (n=705) (n=70) 
MDN T4.1 73.0 62.1 62.0 61.4 65.1 37.3 60.3 66.0 70.6 69.6 68.4 48.9 S15 42.9 52.1 49.5 47.1 


ME/AD 25.8 27.0 S19 28.9 38.6 35.0 27 397340 28.6 30.3 31.6 51.1 49.0 57.2 419505530 


Note: ME and AD were merged because of the smaller group sizes 


Table 9. Comparative Reading Levels for RHR English Learners by Mode and Home Language, HISD, 2016-2017 


Face-to-Face Online 
Reading BOY MOY EOY BOY MOY EOY 
Eevel English Spanish Other English Spanish Other English Spanish Other English Spanish Other English Spanish Other English Spanish Other 
(n=1060) (n=405) (n=47) (n=1146) (n=431) (n=67) (n=1247) (n=615) (n=111) (n = 682) (n= 304) (n= 59) (n=224) (n=116) (n=30) (n=53) (n=480) (n=70) 
MDN 72.3 75.1 74.5 62.4 60.6 65.7 59.1 59.7 64.9 68.8 72.0 84.7 47.8 53.4 40.0 SL3 47.5 61.5 


ME/AD 27.8 25.0 25.5 37.6 39.4 34.3 ee | 31.4 27.9 15.3 52.2 46.6 60.0 Se eae S| 


Note: ME and AD were merged because of the smaller group sizes 
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41) Com 


Proportion of RHR English Learners Fountas and Pinnell Reading Levels Using Repeated Measures, HISD, 2016-2017 


Face-to-Face 


Online 
Reading (n=753) Face-to-Face (n=390) Online 
Level BOY EOY Difference BOY EOY Difference 
MDN 73.8 57.4 70.8 57.8 
ME 17.0 17.4 15.9 18.8 
AD O72 25.2 13.3 19.7 
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Appendix D. RHR Spanish Learners 


Table 11. Proportion of Spanish RHR Learners Reading by Reading Level, Mode and Gender, HISD, 2016-2017 


Face-to-Face Online 
Reading BOY MOY EOY Difference BOY MOY EOY Difference 
Level BOY/EOY BOY/EOY 
Female Male Female Male Female Male Female Male Female Male Female Male Female Male Female Male 
(n=615) (n=703) (n=656) (n=621) (n=687) (n=682) (n=436) (n=338) (n=438) (n=501) (n=470) (n=502) 
MDN 63.9 65.0 48.8 48.1 3L5 54.4 58.7 60.6 46.3 45.1 52.6 51 
ME 19.2 18.2 30.5 29.0 25.9 23.8 28.1 22.7 38.8 39.3 30.0 28.3 
AD 16.9 16.8 20.7 22.9 22.6 21.8 12.2 16.8 14.8 15.6 17.4 20.7 


Table 12. Proportion of Spanish RHR Learners by Reading Levels, Mode, and Economic Status, HISD, 2016-2017 


Face-to-Face Online 
BOY MOY EOY Difference BOY MOY EOY Difference 
BOY/EOY BOY/EOY 
Reading Non-Econ Econ Non-Econ Econ Non-Econ Econ Non-Econ Econ Non-Econ Econ Non-Econ Econ. Non-Econ Econ Non- Econ. 
Level Disadv Disadv Disadv Disadv Disadv Disadv Disadv Disadv Disadv Disady Disadv Disadv Disady Disadv Econ Disadv 
(n=97) (n=1221) (n=107) (n=1170) (n=110) (n=1259) (n=53) (n=771) (n=58) (n=881) (n=75) (n=897) Disadv. 
MDN 67.0 64.3 53.3 48.0 50.8 53.1 56.6 59.8 50.0 45.4 54.7 S15 
ME 16.5 18.8 24.3 30.3 21.8 25.1 24.5 26.2 29.3 39.7 24.0 29.5 
AD 16.5 16.9 22.4 21.7 26.4 21.8 19.8 14.0 20.7 14.9 21.3 19.0 


Face-to-Face Online 
Reading BOY MOY EOY Difference BOY MOY EOY Difference 
Level BOY/EOY BOY/EOY 
Non- At Risk Non- At Risk Non- At Risk Non At Risk Non- At Risk Non- At Risk Non- At Risk Non- At Risk 
At Risk At Risk At Risk At Risk At Risk At Risk At Risk At Risk 
(n=81) (n=1,237) (n=96) (n=1,181) (n=114) (n=1,225) (n=51) (n=773) (n=53) (n=886) (n=53) (n=919 
MDN 67.9 64.3 47.9 48.5 51.8 53.1 62.7 59.4 47.2 45.6 41.5 52.3 
ME 11.1 19.2 25.0 30.1 25.4 24.8 31.4 25.7 35.8 39.3 34.0 28.8 
AD 21.0 16.6 27.1 21.3 22.8 22.2, 5.9 14.9 17.0 15.1 24.5 18.8 
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Table 14. Proportion of RHR Spanish Learners by Reading Levels, Mode, and LEP Status, HISD, 2016-2017 


Face-to-Face Online 
Reading BOY MOY EOY Difference BOY MOY EOY Difference 
Level BOY/EOY BOY/EOY 
Non- LEP Non- LEP Non- LEP Non- LEP Non- LEP Non- LEP Non- LEP Non- LEP 
LEP LEP LEP LEP LEP (n=694) LEP LEP LEP 
(n=236) ——(=1,082)qengs) (n= 1031) (n=262) (n=1,107) (n=130) (n=147) (n=792) (n=142) (n=830) 
MDN 67.4 63.9 46.5 48.9 52.7 53.0 63.8 58.8 41.5 46.5 44.4 53 
ME 16.1 19.2 26.1 30.6 21.0 25.7 26.2 26.1 46.9 37.6 30.3 28.9 
AD 16.5 16.9 27.3 20.4 26.3 21.2 10.0 15.1 11.6 15.9 25.4 18.1 


Table 15. Proportion of Spanish RHR Learners by Reading Levels, Mode, and Ethnicity, HISD, 2016-2017 


Face-to-Face Online 
BOY MOY EOY BOY MOY EOY 
Reading 
Level Black —_ Hispanic White Black Hispanic White Black —_ Hispanic White Black Hispanic White Black Hispanic White Black Hispanic White 
(n=83) (n=1,227) (n=6) (n=83) (n=862) (n=4) (n=72) (n=1,279) (n=10) (n=50) (n=760) (n=7) (n=64) (n=862) (n=9) (n=74) (n=885) (n=6) 
MDN 69.9 64.1 83.3 48.2 48.5 50.0 58.3 52.4 70.0 66.0 58.7 714 40.6 46.2 55.6 47.3 52.4 16.7 


ME/AD 30.1 35.9 16.7 518 51.5 50.0 a ar 34.0 413 28.6 © 59.4 43.8 44.4 eye 


Note: ME and AD were merged because of the smaller group sizes 


Table 16. Proportion of Spanish RHR Learners by Reading Levels, Mode, and Home Language, HISD, 2016-2017 


English Home Language Face-to-Face Online 
BOY MOY EOY BOY MOY EOY 
Reading 
Level English Spanish Other English Spanish Other English Spanish Other English Spanish Other English Spanish Other English Spanish Other 
(n=214) (n=1,103) (n=1) (n=216) (1,058) (n=3) (n=231) (n=1,103) (n=5) (n-114) (n=706) (n=4 (n=133) (n=804 (n=2) (n=130) (n=837) (n=5) 
MDN 65. 64.3 0.0 46.3 48.9 66.7 S11. 53.3 60.0 63.2 58.8 100 42.1 46.4 0.0 44.6 53.0 20.0 


Note: ME and AD were merged because of the smaller group sizes 
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Table 17. Proportion of RHR Spanish Learners Fountas and Pinnell Reading Levels Using Repeated Measures, HISD, 2016-2017 


Spanish Reading Face-to-Face Face-to-Face Online Online 
Level (n=1,108) Difference (n=711) Difference 
BOY EOY BOY EOY 
MDN 63.2 53.2 58.9 51.3 
ME 19.3 26.1 26.7 26.2 
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